Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rolling HDFS upgrade #571

Merged
merged 30 commits into from
Aug 28, 2024
Merged

Rolling HDFS upgrade #571

merged 30 commits into from
Aug 28, 2024

Conversation

nightkr
Copy link
Member

@nightkr nightkr commented Aug 2, 2024

Description

Fixes #362, currently implements the "manual" process from #362 (comment).

Definition of Done Checklist

  • Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant
  • Please make sure all these things are done and tick the boxes

Author

Preview Give feedback

Reviewer

Preview Give feedback

Acceptance

Preview Give feedback

@nightkr nightkr marked this pull request as ready for review August 5, 2024 10:44
@nightkr nightkr requested a review from a team August 5, 2024 10:57
@NickLarsenNZ NickLarsenNZ self-requested a review August 5, 2024 14:38
NickLarsenNZ
NickLarsenNZ previously approved these changes Aug 5, 2024
Copy link
Member

@NickLarsenNZ NickLarsenNZ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just a few typos and non-blocking thoughts and questions

docs/modules/hdfs/pages/usage-guide/upgrading.adoc Outdated Show resolved Hide resolved
docs/modules/hdfs/pages/usage-guide/upgrading.adoc Outdated Show resolved Hide resolved
rust/operator-binary/src/container.rs Outdated Show resolved Hide resolved
rust/crd/src/lib.rs Outdated Show resolved Hide resolved
rust/operator-binary/src/hdfs_controller.rs Outdated Show resolved Hide resolved
rust/operator-binary/src/hdfs_controller.rs Outdated Show resolved Hide resolved
rust/operator-binary/src/hdfs_controller.rs Outdated Show resolved Hide resolved
@NickLarsenNZ NickLarsenNZ self-requested a review August 7, 2024 07:24
NickLarsenNZ
NickLarsenNZ previously approved these changes Aug 7, 2024
Copy link
Member

@NickLarsenNZ NickLarsenNZ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@nightkr
Copy link
Member Author

nightkr commented Aug 9, 2024

Moving back into RfC phase since 10e5220 adds a new .status.upgradingProductVersion CRD field.

@nightkr nightkr requested review from NickLarsenNZ and a team August 9, 2024 15:13
@NickLarsenNZ
Copy link
Member

Moving back into RfC phase since 10e5220 adds a new .status.upgradingProductVersion CRD field.

I didn't see the new field in the CRD YAML.

@nightkr
Copy link
Member Author

nightkr commented Aug 9, 2024

My bad, forgot to make crds. a9809ba

@NickLarsenNZ
Copy link
Member

We could probably move it back to voting to speed it up.
I'll withhold my PR Approval until it's back out of RfC.

@sbernauer
Copy link
Member

TLDR

pub struct HdfsClusterStatus {
    /// The product version that the HDFS cluster is currently running.
    ///
    /// During upgrades, this field contains the *old* version.
    pub deployed_product_version: Option<String>,

    /// The product version that is currently being upgraded to, otherwise null.
    pub upgrading_product_version: Option<String>,
}

Which works for me. WDYT of using upgrade_target_product_version to make it a bit more clear? Or would you consider this too chatty?

@nightkr
Copy link
Member Author

nightkr commented Aug 12, 2024

Sounds like an improvement to me.

@nightkr
Copy link
Member Author

nightkr commented Aug 12, 2024

0604aa6

@nightkr
Copy link
Member Author

nightkr commented Aug 14, 2024

Moving back into voting. Please vote on this comment.

@sbernauer
Copy link
Member

Voting finished, accepted

@nightkr
Copy link
Member Author

nightkr commented Aug 19, 2024

Current merge blockers are:

Copy link
Member

@NickLarsenNZ NickLarsenNZ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments

Cargo.toml Outdated Show resolved Hide resolved
rust/crd/src/lib.rs Show resolved Hide resolved
rust/crd/src/lib.rs Show resolved Hide resolved
rust/crd/src/lib.rs Outdated Show resolved Hide resolved
@NickLarsenNZ NickLarsenNZ self-requested a review August 26, 2024 06:54
NickLarsenNZ
NickLarsenNZ previously approved these changes Aug 26, 2024
Copy link
Member

@NickLarsenNZ NickLarsenNZ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving now so that when the last comment is answered/resolved, this can be merged.

Copy link
Member

@NickLarsenNZ NickLarsenNZ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@nightkr nightkr added this pull request to the merge queue Aug 28, 2024
Merged via the queue into main with commit ae6266d Aug 28, 2024
31 checks passed
@nightkr nightkr deleted the feature/upgrade branch August 28, 2024 11:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support [rolling] upgrade of HDFS
3 participants